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The following statement is a full description of J^ls invention, 
including the best method of performing It known to me/us.- 



AN EFFICIENT METHOD OF MODIFYING AN IMAGE 
P'ield of the Invention 

The present invention relates to manipulation of digital images for display, and in 
particular to applying repairs and effects to images in a manner which enables undoing 
of repairs and the effects to the images in a relatively efficient fashion. 
Background Art 

Electronic photo-albums have become increasingly popular in recent limes. 
These electronic photo-albums typically take the form of a collection of images stored 
using a non-volatile memory device, from which a user can retrieve the image for 
displaying on a display device. Usually, the images are gathered in digital form, and 
may, for example, be scanned by an electronic scanner, down-loaded from the internet 
or produced by a digital camera. A collection of these images can be stored 
ctectronically to create an electronic photo-album, analo^^ous to the traditional family 
photo-album which uses photographic prims. 

Presently, electronic photo-albums lake ihe form of a collection of images, a user 
of the album being able to associate a caption or brief description with each image, l-or 
example, a user can collect a series of family photographs of a wedding, whether lri>m 
a digital video or still camera, and associate a capti(^n such as the dale and a brief 
description of the event with each image. If desired, an elecironic photo-album can be 
reproduced from the memory device by a high quality output device, such as a high 
resolution colour printer, to produce an album substantially similar to a cDnvcntional 
photo-album. 

However, these elecironic photo-albums do not generally provide rnanagcmeni 
capabilities to efficiently experiment with a plurality of special effects and photo-image 
repair. Further, they do not provide for the undoing of the effects and repairs in an 
eftlcieni manner, and in a fasion which ensures the original image is restored. 

It is an object of the invention to overcome, or at least ameliorate, one or more of 
the disadvantages of the prior art. 
Summary of the Invention 

Accordingly, in a first aspect, the invention provides a method of generating a 
plurality of live image data sets, each live image data set representing an image having 
one or more predetermined image effects, said method comprising the steps of: 

providing a raw image representation comprising one or more raster based 
images; 

constructing one or more repaired image representations each being associated 
with the raw image representation and providing a first se» -^f image effects: 



creatine one or more live image representations each being assc^iatcd with at least 
one repaired image representation and each live image representation prov.d.ng a 

second set of image effects; and 

rendering said live image representations, wherein each live image representation 
s associated with a repair image representation inherits the set of f.rst image effects. 

In a second aspect, the invention provides a the apparatus according to paragraph 
11 further comprising means for generating a sequence of image mampulat.on 

coiH lpa<;t one live image data set to be carried out by said 
instructions using said at least one nvc miac. 

rendering processor. 

in a third aspect, the invention provides an apparatus according to any one ot 
paragraphs 11 to 16, farther comprising means for generating raw data coupled to said 
memory to provide one or more raw data sets. 
Brief Description of the Drawings 

An embodiment of the present invention is described hereinafter with rclercnce to 

16 the drawings, in which: 

Figure 1 is a block diagram providing an overview of a preferred embodiment <.l 

inc invcnlion; 

Figure 2 is a block diagram of a general purpose computer upon which the 

invention may be implemented: 

Figure 3 illustrates an example of a relationship between a live image, repaired 
imace and raw image data according to the invention: 

' Figure 4 illustrates an example of a hierarchical structure in executing image 

operations according to the invention: 

Ficure S is a block diauram of a rendering pr.Kcss acc<.rding to the invention: and 
Figure 6 is a schematic flow-chart showing a method of rendering and printing a 

live image, in accordance with the present invention. 

Detailed Description 

-Referring to the drawings, a schematic block representation is illustrated in I igure 
1 to provide In overview of a preferred embodiment of the present invention. Live 
30 image data 100 encapsulates raw image data 101 in a raster based format, and a series 
of image operations that are applied to the raw image data 101. The raw image data 
101 preferablv comprise a plurality of pi.xel-based images produced by a digital video 
or still camera, or image data produced by a digital scanner. For example, a 
photograph taken by a conventional still camera can be scanned into a memory storage 

35 device using a digital scanner. 

The image operations 102 apply repairs and/or special effects to the raw image 
data 101. A "repair effect may include such operations as ■red-eye" removal (often 
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caused by flash photosraphy). scratch removal, dust removal, colour correction, and 
unsharp f.ltering, amongst others. A special effect may include such operations as scp.a 
effect monochrome effect, mirror reflection, rotation, cropping or res.z.ng. and 
blurring effects, amongst others. A displayed image (ie. a pixel representation) ot the 
live image data is, preferably, not realised until the live image data is rendered. The 
technique adopted for the preferred embodiment of the invention involves constructmg 
live image data (and preferably a realised pixel representation) by applymg one or more 
operations to the raw image data 101 in such a manner that the raw image data 101 
remains unaltered. 

The invention is practised using a conventional general -purpose computer, such as 
that shown in Figure 2, wherein the image manipulation is implemented as soltwarc 
executed on the computer. The computer system 200 comprises a computer 201. mput 
devices 202, 203 and a display device 204. 

The computer 201 comprises at l.ast one processor unit 205. a memory un.t 200 
which typically includes random access memory (RAM) and read only memory (ROM), 
and input/output (I/O) interfaces including a video interface 207. and an interface 20« 
for a keyboard 202. an input digital scanner device 211 and mouse «.r joyst.ck) 203. 
The storaae device 209 can include one or more of the following devices: a floppy d.sk. 
a hard disk drive, a CD-ROM drive or similar non-volatile storage devices known lo 
those skilled in the art. Further, the general-purpose computer can have additional I/O 
interfaces 212 for such peripheral devices as a printer 213 and/or a photographic (v.dco 
or still) camera 214 for downloading images to the computer. The components 205 t.> 
^09 and 212 of the computer 201 typically communicate via an interconnected bus 210. 
and in a manner which results in a usual mode of operation of the c<.mputer sys.e.n 200 
known to those skilled in the relevant art. Fxamples of computers on which the 
embodiment can be practised include IBM-FC/ATs and compatibles. .Sun Sparcstat.ons 

or like computer systems. 

Turning to Fiaure 3. there is shown a schematic diagram of an example of a 
plurality of live imaee data sets (reference numerals 300 to 302) associated with a set of 

, raw image data 304 representing a single raster image. Arrows with solid heads 
indicate Hnes associated with reference numerals, whilst the arrows with non-solid 
heads indicate referencing of repaired image data and raw image data in an image 
register 303. which is discussed below. A first set of repaired image data 305 
comprising a first sequence of image operations 306 is shown, together with a second 

5 set of repaired imaee data 307 comprising a second series of image operations 308. 
Associated with the first set of repaired image data 305 are two sets of live image data 
(300 301). each comprising a sequence of additional image operations (309. 310) 
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luriher to the image operation 306 of the associated set ot repaired image data 305. 
The third set of live image data 302. also comprising a series of operations 311. is 
associated with the set of second repaired image data 307. 

Each live image data set (301. 302) represents, when rendered, a pi.xel based 
image having raw image data 304 in common and having different repairs and special 
effects applied. Two of the sets of live image data (301 and 302) have the first set of 
repaired image data 305 in common. Therefore, a pixel-based image resulting from 
each of the two sets of live image data (301, 302) share repair and/or effect operations. 

An image register 303 tracks repaired image data (305. 307) and evaluates 
whether the repair image data is still in use by a live image data (items 300 to 302). 

This embodiment of the invention allows for a plurality of .sets of repaired image 
data for each raw image data, and for a plurality of sets of live image data for each set 
of repaired image data. In this manner, a plurality of .sets of live image data can share a 
set of repaired image data and therefore have substantially the same repair effected on 
each one of the sets of live image data sets. The live image data may be rendered lo a 
pixel repre.sentation for display on a display device 204 (or a printer 213). and each set 
of live image data represents one image in the pixel repre.sentation. A u.ser of the 
.system manipulates the live image data by applying (or ^electing) desired image 
operations on the pixel representation. For example, a user may desire to rotate a 
displayed image associated with a live image data set which in turn is associated with a 
repaired miage data. The user selects a rotation operation lor the displayed image, 
which is then inserted as an operation instruction into the live image data set and a 
corresponding rotated displayed image is generated. Preferably a pixel repre.sentation 
of the repaired image data or raw image data is not realised to the display device 204. 

If an operation is to be applied to pixel representations of live image data 
associated with a set of repaired image data, the operation is in.serted in the set of 
repaired image data. This results in all image data associated with that .set of repaired 
image data to inherit the effects of the operation. For example, a "red-eye" removal 
operation can be performed on a plurality of pixel representations of live image data 
associated with a particular set of repaired image data, by inserting the red-eye removal 
operation as an instruction sequence of the repaired image data. 

Repair operations and special effects operations affecting a single image are 
inserted as operation instructions into live image data corresponding to the single 
image. Repair operations and special effects operations affec-ine a plurality of images 
associated with sets of live image data, which are i. .urn associated with a single set of 
repaired image data, are inserted as operation instructions into the single set of repaired 
image data. Consequently, changes to a pixel representation affected by a repair 
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operation or special effect operation can be • undone" by removing the corresponding 
operation instruction from a live image data associated with the pixel representation, 
and then realising a pixel representation from the live image data with the removed 
operation. Accordingly, removing an operation instruction from a set of repaired image 
data will "undo" the operation for any associated sets of live image data. Thus, the 
described arrangement a relatively efficient procedure for undoing repair and special 
effect operations applied to one or more images. 

Further, providing a single repository of raw image data for a plurality of sets of 
image data from which,a plurality of images (pixel representation) can be rendered 
reduces storage requirements. 

Preferred components of the invention are described separately hereinafter. 

1. Raw Image data 

Raw image data consists of raster image data for one or more images, each raster 
image data preferably being stored, in a single repository (a non-volatile storage 
means), as both high and low resolution data. 

Preferably, when a live image is rendered to a pi.xel representation lor a display 
device, the low-resolution data of the raw image data is used, conver.sely. when the live 
imace data is rendered to a printing device, the high resolution data is used. Utilising 
the low resolution data provides a speed advantage in rendering, which is particularly 
u.seftjl for display devices whilst using the high resolution data provides enhanced 
quality for the rendered image, which is desirable for printing applications. 

2. Repaired Image Data 

Repaired image data includes a link in the form of a reference pointer to raw 
image data and a sequence or list of repair operations and/or special etfecis operations 
to be applied to the raw image data. 

3. Live Image Data 

Live image data includes a link in the for reference pointer to a repaired image 
data and a set or series of repair and/or special effects operations (repairs and/or special 
effects operations) that are to be applied to raw image data in addition to operations 
associated with the repaired image data. Operations are inserted to and removed from 
the live image data. When an operation is to affect all live im^.-e data associated with a 
repaired image data the operation is insened into the repaired image data, and typically 
passed down to the repaired image data via the live image data. 

4. Image Register 

An image register maintains a reference pointer to each set of repaired image data 
and raw image data loaded in memory (which preferably is Random Access Memory 
(RAM)). A plurality of reference counters are also maintained by the image register. 
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which .rack how many sets of hv. image dab. a,Ki se.s of rcpa.r ,n,as= d:„a. ar. 
Tss^l-ed <v,a reference pcin.ers, wi.h a raw image da,a. The re.erence counu-r. 
m a e * sharin. of images .hrough reference counting. For example, .wo sets o, 
W im e data sh^are a se, of repaired image da.a. .hen a corresponding re erence 

„ er as a value of .wo. If .he ,wo >ive image da.a se.s are deie.ed ..yp.ca y hy 
uler, ,he reference counter drops .0 zero. Since .he correspond.ng repa.red .mage d^a 
Tno linger in use. i, is removed from memory^ maR.ng .h-- memory ava.lahle. In .h. 
IZ .h= ™age register enables memory .o be reCa.med when a se, o. repa.rcd ,mag. 
dMta or a raw image data is no longer used. 
^ Imaec Operations 

AS previouslv described, image opera.ions nn,y include one or more repair 
opera.ions and one' or more special effec.s opera.ions. Preferably, i'™--' 
mser.ed in.o se.s of live i.nage da.a are special effec.s opc.ra„o„s. wh,l , r.pa 
opera.ions are preferably inser.ed m,o se.s of repaired in.age da.a. However, .he un . 

opera.ions ,nay also be ,nser,ed naer-changeably ,n .he se.s o, „ve unage da, ,hc 

sets of" repair image data. 

. Kcir-.. i . liss structure and do not themselves 
imaoe operations torm pari ol an abstract class structure 

in^plemenl repairs or special e.Tects on an image. To .mp.en.ent an .mage operation^ an 

instruction routine is imp.emen.ed together with predetermined m.ormation .o generate 

an imaee manipulation instruction set. I-or example, a rotate operat.o.i rcMUires 

predete;m.ned information leg. angle o, rotation, and an msirucuon n>utine .or he 

rotation operation that genera.es a set of image manipulation instructions lo rotate the 
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Turnin. to Rizure 4. there is shown an example o. a hierarchical da.a structure o 
in.aoc operanons in accordance w.th a preferred en,bodiment. Live image data 4.)() 
.ncorporates a rotation operation 401 and a reference pointer 402 u. repaired image data 
403 The rotation operation has associated predetermined information 404 (eg. angle ol 
rotation). The repaired image data 403 incorporates a • red-eye" removal operation 407 
and a reference pointer 405 to a raw image data 406 comprising a pixel-based image. 
The red-eye removal operation 407 has associated therewith predetermined mformation 
408 relaiinc to the operation (eg. red-eye removal data,. 

When the live imase data 400 is rendered to a pixel representation, preferably tor 
printing or displav. red-eye removal 407 is performed m conjunction w.th the raw 
.mace data. A rotation operation 401 is subsequently performed to result in a rendered 
pixd representation of the raw image data that has been red-eye corrected and rotated as 
desired If a user chooses to undo the rotation operation 401 . the rotation operation 40 
is removed from the live image data 400 and the reference pointer 402 to the repaired 
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iniaee data 403 is maintained. The live image data 400. with the rotation operation 401 
removed, is re-rendered to produce a pixel representation of the raw image data having 
been red-eye corrected only, 

A render process for producing a pixel representation of a live image involves 
applying each operation in turn to raw image data. The render process consists of two 
stages, being: 

(1) production of image manipulation instructions: and 

(2) execution of the image manipulation instructions to produce a pixel representation 
in the form of a raster-based image. 

At the first stage, instructions are recorded in a data stream to be executed during 
the second stage. The data stream may consist of a data file containing instructions or a 
data structure in memory representing a series of instructions for performing an 
operation. 

The render process begins by requesting a most recent operation inserted into a 
live image data to generate an instruction sequence. The insiruciiiMi sequence 
recursively calls an operation that was inserted into the live image data before a current 
operation. This recursive activity forms a chain of calls from most recent operation to 
least recent operation, with each op^eration generating a corresponding instruction 
sequence. When a iast live image data operation is reached, a call is generated for a 
most recent operation of the repaired image data. The generation of an instruction 
sequence for each operation of the repaired image data is described in a substantially 
similar manner as for the generation of the instruction sequence for each i)peratii>n of 
the live image data referred to above. When the last operation of the repaired image 
data is called, an instruction to import raw image data is added to the data stream. 

The second stage involves executing the data stream, and is typically a more 
computationally intensive process in comparison to the instruction sequence generation 
described above. The data stream of image manipulation instructions can be siored and 
executed by a predetermined render processor. The data stream of image manipulation 
instructions can therefore optionally be rendered on a separate render processor either 
concurrently or remotely. 

Referring to Fig 5. there is shown a render process according to the preferred 
embodiment. An instruction sequence 500 is generated for substantially all operations 
incorporated in live image data and repaired image data (ie. a data stream is generated *. 
The data stream is stored on a storage means 501 (preferably a non-volatile memory 
device such as a hard drive) for execution by a render processor 502 to provide a pixel 
representation 503. 
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Turning to Figure 6. there is shown a flowchan setting out the basic steps in the 
rendering process. These steps would take place in a computer, such as that described 
above. Raw image data from a digital camera, scanner or the like is initially input to 
the computer, and a live image data structure created (block 601) in accordance with the 
steps outlined above. The resultant set of live image data is then rendered to the screen 
(block 602) for viewing by a user. If the users wishes lo make modifications, 
operations are removed (block 604) from the relevant set of live image data, the 
changes saved to disk (block 606) and the image again rendered to the screen (bKxrk 
602). When the user is happy with the displayed result, live image data set is rendered 
to a printer (block 607). The user may also restore or retrieve a saved set of live image 
daiy from disk (block 608), which is then rendered to the screen for viewing (bhvk 
602). Further modifications may then be made or the image rendered to the piinter. as 
described above. 

Although the inventii)n has been described with reference ti> a numb"r of 
particular ex;'.:nples. it will be appreciated by iht>se skilled in the an thai the invcniii>n 
may be 'embodied in many (>iher forms. 
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The claims defining the invention are as follows: 

1 A method of generating a plurality of live image data sets, each live image data 
SCI representine an image having one or more predetermined image effects, said method 

comprising the steps of: 

providing a raw image representation comprising one or more raster based 

images: 

constructing one or more repaired image representations each being assiKiaied 
with the raw image representation and providing a first set of image effects: 

creating one or more live image representations each being associated w uh at lea^l 
one repaired image representation and each live image representation providing a 
second set of image effects: and 

rendering said live image representations. v» herein each live image represeniaiion 
asstKiaied with a repair image representation inherits the set of first image effects. 

2- The method as recited in claim I. wherein the tirsl image effects ci . uprise repair 
effects. 

The method as recited i:i viaim 1, vv.:erein the second iina^e eftecis comprise 
special effects. 

4. The method as recited in claim 1. wherein the rau image repre^enlalion comprises 
a plurality of resv>Iulions of a raster based image. 

5. The methixl as recited in claim 4. wherein the plurality i>f resolutions c<»n:prises a 
high and low resolutitMi of a raster based image. 

6. A method i>f providing a hierarchical structure for image iiianipulatit>n. said 
method comprising the steps of: 

creating a repaired image comprising a first list of image manipulation operations; 

creating one or more live images, eac'i live image comprising a different list of 
image manipulation operations: and 

associating said repaired image and the one (/r more live images to pr*iuuce a 
hierarchical structure wherein each live image ass(x:iaied with the repaired image 
inherits the first list of image manipulation operations. 
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7. The method as recited in claim 6, wherein the first list and the different lisi of 
image manipulation operations are furfher provided with a capability of removing or 
inserting one or more of said image manipulatic n operation. 

8. The method as recited in c?aim 6, wherem the image manipulation operations 
comprise operation for producing repair effects and/or special effects. 

9. The method as recited in claim 6. wherein sa i hierarchical structure for image 
manipulation is applied to raw iinage data and a resuU of the application is rendered to 
proviae one or more pixel (raster image) represeniations. 

10. An apparatus for manipulating images in accordance with any one of the 
preceding claims. 

1 1 . An apparatus for rendering a digital output image to he displayed u^^ini: :i 
reproduction device, said apparatus comprising: 

a rendering pro<:essor for generating <aid digital output image. 

a memory coupled to said rendering processor: 

raw image data comprising a plurality of pixels, wherein said raw irnai:c data is 
stored in said memory: 

means for generating at least one repaired image data set. each such rep;iiicd 
image data set comprising linking means referencing said raw image data and one *>r 
more operation instructions for nnxlifying said raw image data. 

means for generating at least i>ne live image data set for ca':h suet: rep;iired image 
data set, each such live image data se: comprising linking means refercnemi: each such 
repaired image data set and one *>r more operation insirueiii»ns f-.)r m(>vlif\ini: ^*:id 
respective repaired image data: 

wherein said rendering processor generates said digital output image dependent *>n 
each of said at le?si one live image data set. 

12. The apparatus according to claim 1 1. further comprising means for generating a 
sequence of image manipulation instructions using said at least one live imaLc data set 
to be carried out by said rendering pr(>:essor. 

13. The apparatus according to claim 11. further comprising an image register me;ins 
for storing a reference to said raw image data and a reference to each such repaired 
image data set. 
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14. The app^ atus according to claim 13. further comprising means for counting said 
references and for removing said raw image data or said .epaired image data set from 
memory when said raw image data or said repaired image data «-et has zero 
corresponding references. 

15. The apparatus according to claim 1 1 . wherein said image forms at least a portion 
of an electronic photo-album. 

16. The apparatus according to any one of claims 1 1 to 15. wherein said raw data 
comprises two sets of pixel data, wherein a first set of said pixel data has a first 
resolution for reproduction us^ng a video display means and a second set of said pixel 
data has a second resolution for reproduction using a printing device, wherein said first 
resolution is lower than said second resolution. 

17. The apparatus according to any one of claims 1 1 lo 16. further comprising means 
for generating raw data coupled to said memory lo provide one or more raw data sets. 

18. The apparatus according to claim 1 1 . wherein .said one or more modifying 
operations of each such repair image data set vre repair operations. 

19. The apparatus according to claim 1 1 . wherein .said one or more modifying 
operations of each such live image data set are special effect operations. 

20. The apparatus according to claim 1 1 . further comprising means for selecting one 
or more minJifying operations and storing a corresponding modifying operation 
instruction in each such repair image data set or live image data set. 

21. A method of generating one or more live images, the method including the steps 
of: 

providing a raster-based raw image representation: 

generating a repaired image representation, based on the raw image representation 
and one or more selected first image effects from a first set of image effects: 

generating a live image representation, based on :he repaired image .epresentation 
and one or more selected second image effects from a second set of image effects, such 
that said live image representation incorporates the selected first and second image 
effects: and 



rendering said live image representation. 



22. A method of generating one or more live images according to claim 21, wherein 
the first set of image effects substantially comprises repair effects. 

23. A method of generating one or more live images according to claim 21 or claim 
22, wherein the second set of image effects substantially comprises special effects. 

24. A method of generating one or more live images according to claim 21, wherein 
each raw image representation includes a raster-based image represented at a plurality 
of resolutions. 

25. A method of generating one or more live images according to claim 24, wherein 
the raster-based image is represented at a high resolution and a low resolution. 

26. A method of generating one or more live images according to any one of the 
preceding claims, v/herein the repaired image representation includes a first list of data 
manipulation operations. 

27. A method of generating one or more live images according to claim 26, wherein 
the repaired image representation includes a pointer to the raw image reprcscnlalion. 

28. A method of generating one or more live images according to any one of the 
preceding claims, wherein the live image represeniaiion includes a second list of data 
manipulation operations. 

29. A method of generating une or more live images according to claim 28, wherein 
the live image representation includes a pointer to the repaired image rcpresenlaiion. 

30. A method of generating one or more live images according to any one of the 
preceding claims, wherein the raw image repre.sentation is ass(>ciaied with a plurality of 
repaired image representations. 

31. A method of generating one or more live images according to claim 30. wherein 
at least one of the repaired image representations is associated with a plurality of live 
image representations. 
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32. A method of creating a hierarchical data structure, the method comprising the 

steps of: 

providing a raw image; 

generating a repaired image based on the raw image, the repaired image including 
a first list of image manipulation operations; 

generating one or more live images based on the repaired image, each live image 
including a second list of image manipulation operations distinct from the first list of 
image manipulation operations; and 

associating the repaired image and the live images to produce a hierarchical 

structure. 

33. A method of creating a hierarchical data structure according to claim 32, further 
including the step of inserting or deleting one or more of the image manipulation 
operations into or from the first or second lists. 

34. A method of creating a hierarchical data .structure according to claim 32 or claim 
33, wherein the image manipulation operations include repair effects or special effects. 

35. A method of creating a hierarchical data structure according to claim 34. wherein 
the first li.st includes repair effects, and the second list includes special effects. 

36. A method of creating a hierarchical data structure according to claim 3 1 . wherein 
the live images are rendered to provide raster-ba.sed pixel images for display. 

37. An apparatus for rendering an output image for display on a reproduction device, 
the apparatus including: 

a rendering processor for generating said output image; 

a memory coupled to said rendering processor, the memory containing raw image 
data comprising a plurality of pixels; 

first means for generating a set of repaired image data based on the raw image 
data, the set of repaired image data including a first linking means which references the 
raw image data and one or more operation instructions for modifying the raw image 
data; 

second means for generating at least one set of live image data, each of the .sets of 
live image data being based on a repaired image data set and including a second linking 
means which references the set of repaired image data and one or more operation 
instructions for modifying the set of repaired image data; 



wherein said rendering processor generates each output image based on one of the 
sets of live image data. 

38. An apparatus for rendering an output image for display on a reproduction device 
according to claim 37, further comprising third means for generating a sequence of 
image manipulation instructions based on one or more the sets of live image data, the 
image manipulation instructions being configured for execution by the rendering 
processor. 

39. An apparams for rendering an output image for display on a reproduction device 
according to claim 38, further including an image register for storing a reference to said 
raw image data and a reference to each set of repaired image data. 

40. An apparatus for rendering an output image for display on a reproduction device 
according to claim 39, further comprising counting means for counting said references 
and removing said raw image data or said set of repaired image data from memory 
when said raw image data or said repaired image data set has zero corresponding 
references. 

41. An apparatus for rendering an output image for display on a reproduction device 
according to claim 37, wherein the output image forms part of an electronic photo- 
album 

42. An apparatus for rendering an output image for display on a reproduction device 
according to any one of claims 37 to 41, wherein the raw data includes two sets of pixel 
data, the first set of pixel data being at a first resolution for reproduction by a video 
display, and the second set of pixel data being at a second resolution for reproduction 
using a printing device, said first resolution being lower than said second resolution. 

43. An apparatus for rendering an output image for display on a reproduction device 
according to any one of claims 37 to 42, further including fourth means connected with 
the memory to generate raw data, thereby to provide one or more sets of raw data. 

44. An apparatus for rendering an output image for display on a reproduction device 
according to claim 37, wherein one or more of the operation instructions associated 
with each set of repaired image data are repair operations. 



OiQLab03 37000.1R rFPOR7-tAn 



10 



15 



20 



15 - 



45. An apparatus for rendering an output image for display on a reproduction device 
according to claim 37. wherein one or more of the operation instructions associated 
with each set of live image data are special effect operations. 

46. An apparatus for rendering an output image for display on a reproduction device 
according to claim 37, further including fifth means for selecting one or more desired 
special effects or repair effects, and storing a corresponding operation instruction in the 
set of repaired image data or one of the sets of live image data. 

47 . A method of generating a plurality of sets of live data, each set of live image data 
having one or more predetermined image effects associated with it, the method being 
substantially as herein described with reference to any one of the embodiments of the 
invention shown in the accompanying drawings. 

48. A method of generating a hierarchical data structure substantially as herein 
described with reference to any one of the embodiments of the invention shown in the 
accompanying drawings. 

49. An apparatus for rendering an output image for display on a reproduction device, 
the apparatus being substantially as herein described with reference to any one of the 
embodiments of the invention shown in the accompanying drawings. 

DATED this Twentieth Day of February 1997 
Canon Kabushiki Kaisha 
Patent Attorneys for the Applicant 
SPRUSON & FERGUSON 



ABSTRACT 
An Efficient Method of Modifying an Image 



A method and apparatus for generating one or more live images. Initially, a raster- 
based raw image representation is provided. A repaired image representation is 
generated, based on the raw image representation and one or more selected first image 
effects from a first set of image effects. Next, a live image representation is generated, 
based on the repaired image representation and one or more selected second image 
effects from a second set of image effects, such that said live image representation 
incorporates the selected first and second image effects. Finally, the live image 
representation is rendered to a display, printer or the like. 
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